In image processing, an adaptive loop filter (ALF) is performed to remove or reduce quantization noise and artifacts introduced during compression through wiener filtering as shown in FIG. 1. On/off signaling is able to be performed using a block-based adaptive loop filter (BALF) or a quad-tree based adaptive loop filter (QALF).
In quad-tree based on/off signaling, for each block (except those in the lowest layer), 1 bit is sent to specify if it is partitioned or not (partitioning signaling). For each non partitioned block (blocks in the lowest layer are always not partitioned), 1 additional bit is sent to specify if the ALF is applied to the block (on/off switch signaling). Given the Wiener filter, the quad-tree structure is optimized using bottom up recursive decision as shown in FIG. 2.
A Wiener filter is trained online based on the statistics of the pixels that have the ALF on. Training is done by the encoder. In some embodiments, each frame has a Wiener filter, of which the filter coefficients are sent to the decoder. Training of Wiener filters requires knowledge of the on/off information of the pixels. However, the on-off decision also requires knowledge of the Wiener filter.
The iteration decision includes training an initial Wiener filter (e.g. based on all pixels), deciding which position to place an on/off switch based on the current Wiener filter, updating the Wiener filter based on all “on” pixels, and performing deciding and updating multiple times (e.g. until conversion).
There are many drawbacks of ALF. In the encoder, there is high computation complexity since there are multiple passes of optimization in filter training and on/off switching. The encoder requires additional delay of one frame (without ALF, the delay is in the order of lines). The encoder stores/retrieves the cross- and auto-correlation matrices for each leaf node multiple times meaning significant memory accesses. The bitstream includes overhead bits signaling the filter coefficients. In the decoder, the Wiener filter coefficients are variable, and multiplication between two variables is expensive compared to multiplying a variable by a fixed number which is able to be replaced with several bit-shifts and additions. Additionally, the hardware implementation of these ALF techniques is very costly, particularly for real-time encoding, because training of Wiener filters has to be done after the entire frame is encoded, and the results of ALF are needed for motion estimation of the next frame. Therefore, the entire pipeline is prolonged, which requires a higher clock rate and more power consumption to meet the real-time constraint.